A Deduplication Study for Host-side Caches with Dynamic Workloads in Virtualized Data Center Environments

Deduplication is a well-known method that improves storage efficiency and reduces the cost of storage in corporate data centers [3, 4]. For virtualized data centers, and in particular for virtual desktop infrastructure (VDI), centrally-managed networked storage can greatly reduce the overall data footprint because virtual machine (VM) disk images have largely the same content. Recent work by Byan et al. [1] suggested employing flash memory-based host-side caches inside VM hypervisors to shed load from the shared storage infrastructure. They demonstrated that such caches can be effective for read-mostly workloads with stable working sets. However, it is not known if they can be as effective in a dynamic environment, when virtual machines migrate frequently from one physical server (hypervisor) to another or when their working sets change. First, since these caches are large (many 100s of GBs), re-warming them with the active content to reach a steady state cache hit rate after a VM migrates to another hypervisor can take as much as 12 hours [1]. Second, as each VM disk image is a separate entity, the caches might contain many copies of the same content even though the network-attached shared storage system would store only a single instance, unnecessarily polluting the host-side hypervisor caches and reducing their overall cache hit rate. The goal of our study is to explore the effectiveness of deduplication for large host-side caches in virtualized data center environments running dynamic VDI workloads. To that end, we analyze traces captured from VDI deployments as well as general enterprise workloads [2]. Understanding the intrinsic properties of data duplication can help us design effective host-side cache policies. Previous deduplication systems and studies focused on data mostly at rest such as backups [4] and archives or on reducing network traffic across a WAN link. Our study on eight traces in duration from two minutes to almost 100 days shows that deduplication can reduce the data footprint inside host-side caches by as much as 67%. This allows for caching a larger portion of the data set and improves the effective cache hit rate. More importantly, such increased caching efficiency can alleviate load from networked storage systems during I/O intensive workloads when most VM instances perform the same operation such as virus scans, OS patch installs (a.k.a. update storms), and reboots (a.k.a. boot storms).